Neural networks have recently been employed as material discretizations within adjoint optimization frameworks for inverse problems and topology optimization. While advantageous regularization effects and better optima have been found for some inverse problems, the benefit for topology optimization has been limited. Previously, the majority of investigations has focused on the compliance problem. We demonstrate how neural network material discretizations can, under certain conditions, find better local optima in more challenging optimization problems. We present the specific case of acoustic topology optimization under periodic excitations, that we model with the Helmholtz equation. We demonstrate that the chances of identifying a better optimum can be improved by running multiple partial optimizations with different neural network initializations. Furthermore, we show that the advantage of using a neural network for the discretization of the material field comes from the interplay with the Adam optimizer. Additionally, we point out the current limitations of the approach compared to constrained and higher-order optimization techniques.